Approaches to Combining Local and Evolutionary Search for Training Neural Networks: a Review and Some New Results

Training of neural networks by local search such as gradient-based algorithms could be diicult. This calls for the development of alternative training algorithms such as evolutionary search. However, training by evolutionary search often requires long computation time. In this chapter, we investigate the possibilities of reducing the time taken by combining the eeorts of local search and evolutionary search. There are a number of attempts to combine these search strategies, but not all of them are successful. This chapter provides a critical review of these attempts. Moreover, diierent approaches to combining evolutionary search and local search are compared. Experimental results indicate that while the Baldwinian and the two-phase approaches are ineecient in improving the evolution process for diicult problems, the Lamarckian approach is able to speed up the training process and to improve the solution quality. In this chapter, the strength and weakness of these approaches are illustrated, and the factors aaecting their eeciency and applicability are discussed.